Palette-colouring: a belief-propagation approach

We consider a variation of the prototype combinatorial-optimisation problem known as graph-colouring. Our optimisation goal is to colour the vertices of a graph with a fixed number of colours, in a way to maximise the number of different colours present in the set of nearest neighbours of each given vertex. This problem, which we pictorially call "palette-colouring", has been recently addressed as a basic example of problem arising in the context of distributed data storage. Even though it has not been proved to be NP complete, random search algorithms find the problem hard to solve. Heuristics based on a naive belief propagation algorithm are observed to work quite well in certain conditions. In this paper, we build upon the mentioned result, working out the correct belief propagation algorithm, which needs to take into account the many-body nature of the constraints present in this problem. This method improves the naive belief propagation approach, at the cost of increased computational effort. We also investigate the emergence of a satisfiable to unsatisfiable "phase transition" as a function of the vertex mean degree, for different ensembles of sparse random graphs in the large size ("thermodynamic") limit.Comment: 22 pages, 7 figure

On Cavity Approximations for Graphical Models

We reformulate the Cavity Approximation (CA), a class of algorithms recently introduced for improving the Bethe approximation estimates of marginals in graphical models. In our new formulation, which allows for the treatment of multivalued variables, a further generalization to factor graphs with arbitrary order of interaction factors is explicitly carried out, and a message passing algorithm that implements the first order correction to the Bethe approximation is described. Furthermore we investigate an implementation of the CA for pairwise interactions. In all cases considered we could confirm that CA[k] with increasing $k$ provides a sequence of approximations of markedly increasing precision. Furthermore in some cases we could also confirm the general expectation that the approximation of order $k$, whose computational complexity is $O(N^{k+1})$ has an error that scales as $1/N^{k+1}$ with the size of the system. We discuss the relation between this approach and some recent developments in the field.Comment: Extension to factor graphs and comments on related work adde

Dynamic rewiring in small world networks

We investigate equilibrium properties of small world networks, in which both connectivity and spin variables are dynamic, using replicated transfer matrices within the replica symmetric approximation. Population dynamics techniques allow us to examine order parameters of our system at total equilibrium, probing both spin- and graph-statistics. Of these, interestingly, the degree distribution is found to acquire a Poisson-like form (both within and outside the ordered phase). Comparison with Glauber simulations confirms our results satisfactorily.Comment: 21 pages, 5 figure

Fast Decoders for Topological Quantum Codes

We present a family of algorithms, combining real-space renormalization methods and belief propagation, to estimate the free energy of a topologically ordered system in the presence of defects. Such an algorithm is needed to preserve the quantum information stored in the ground space of a topologically ordered system and to decode topological error-correcting codes. For a system of linear size L, our algorithm runs in time log L compared to L^6 needed for the minimum-weight perfect matching algorithm previously used in this context and achieves a higher depolarizing error threshold.Comment: 4 pages, 4 figure

Statistical-mechanical iterative algorithms on complex networks

The Ising models have been applied for various problems on information sciences, social sciences, and so on. In many cases, solving these problems corresponds to minimizing the Bethe free energy. To minimize the Bethe free energy, a statistical-mechanical iterative algorithm is often used. We study the statistical-mechanical iterative algorithm on complex networks. To investigate effects of heterogeneous structures on the iterative algorithm, we introduce an iterative algorithm based on information of heterogeneity of complex networks, in which higher-degree nodes are likely to be updated more frequently than lower-degree ones. Numerical experiments clarified that the usage of the information of heterogeneity affects the algorithm in BA networks, but does not influence that in ER networks. It is revealed that information of the whole system propagates rapidly through such high-degree nodes in the case of Barab{\'a}si-Albert's scale-free networks.Comment: 7 pages, 6 figure

Gaussian Belief with dynamic data and in dynamic network

In this paper we analyse Belief Propagation over a Gaussian model in a dynamic environment. Recently, this has been proposed as a method to average local measurement values by a distributed protocol ("Consensus Propagation", Moallemi & Van Roy, 2006), where the average is available for read-out at every single node. In the case that the underlying network is constant but the values to be averaged fluctuate ("dynamic data"), convergence and accuracy are determined by the spectral properties of an associated Ruelle-Perron-Frobenius operator. For Gaussian models on Erdos-Renyi graphs, numerical computation points to a spectral gap remaining in the large-size limit, implying exceptionally good scalability. In a model where the underlying network also fluctuates ("dynamic network"), averaging is more effective than in the dynamic data case. Altogether, this implies very good performance of these methods in very large systems, and opens a new field of statistical physics of large (and dynamic) information systems.Comment: 5 pages, 7 figure

Sums over geometries and improvements on the mean field approximation

The saddle points of a Lagrangian due to Efetov are analyzed. This Lagrangian was originally proposed as a tool for calculating systematic corrections to the Bethe approximation, a mean-field approximation which is important in statistical mechanics, glasses, coding theory, and combinatorial optimization. Detailed analysis shows that the trivial saddle point generates a sum over geometries reminiscent of dynamically triangulated quantum gravity, which suggests new possibilities to design sums over geometries for the specific purpose of obtaining improved mean field approximations to $D$-dimensional theories. In the case of the Efetov theory, the dominant geometries are locally tree-like, and the sum over geometries diverges in a way that is similar to quantum gravity's divergence when all topologies are included. Expertise from the field of dynamically triangulated quantum gravity about sums over geometries may be able to remedy these defects and fulfill the Efetov theory's original promise. The other saddle points of the Efetov Lagrangian are also analyzed; the Hessian at these points is nonnormal and pseudo-Hermitian, which is unusual for bosonic theories. The standard formula for Gaussian integrals is generalized to nonnormal kernels.Comment: Accepted for publication in Physical Review D, probably in November 2007. At the reviewer's request, material was added which made the article more assertive, confident, and clear. No changes in substanc

Belief propagation algorithm for computing correlation functions in finite-temperature quantum many-body systems on loopy graphs

Belief propagation -- a powerful heuristic method to solve inference problems involving a large number of random variables -- was recently generalized to quantum theory. Like its classical counterpart, this algorithm is exact on trees when the appropriate independence conditions are met and is expected to provide reliable approximations when operated on loopy graphs. In this paper, we benchmark the performances of loopy quantum belief propagation (QBP) in the context of finite-tempereture quantum many-body physics. Our results indicate that QBP provides reliable estimates of the high-temperature correlation function when the typical loop size in the graph is large. As such, it is suitable e.g. for the study of quantum spin glasses on Bethe lattices and the decoding of sparse quantum error correction codes.Comment: 5 pages, 4 figure

Probabilistic Bag-Of-Hyperlinks Model for Entity Linking

Many fundamental problems in natural language processing rely on determining what entities appear in a given text. Commonly referenced as entity linking, this step is a fundamental component of many NLP tasks such as text understanding, automatic summarization, semantic search or machine translation. Name ambiguity, word polysemy, context dependencies and a heavy-tailed distribution of entities contribute to the complexity of this problem. We here propose a probabilistic approach that makes use of an effective graphical model to perform collective entity disambiguation. Input mentions (i.e.,~linkable token spans) are disambiguated jointly across an entire document by combining a document-level prior of entity co-occurrences with local information captured from mentions and their surrounding context. The model is based on simple sufficient statistics extracted from data, thus relying on few parameters to be learned. Our method does not require extensive feature engineering, nor an expensive training procedure. We use loopy belief propagation to perform approximate inference. The low complexity of our model makes this step sufficiently fast for real-time usage. We demonstrate the accuracy of our approach on a wide range of benchmark datasets, showing that it matches, and in many cases outperforms, existing state-of-the-art methods

Dynamic mean-field and cavity methods for diluted Ising systems

We compare dynamic mean-field and dynamic cavity as methods to describe the stationary states of dilute kinetic Ising models. We compute dynamic mean-field theory by expanding in interaction strength to third order, and compare to the exact dynamic mean-field theory for fully asymmetric networks. We show that in diluted networks the dynamic cavity method generally predicts magnetizations of individual spins better than both first order ("naive") and second order ("TAP") dynamic mean field theory